Learning Concise Models from Long Execution Traces

Abstract models of system-level behaviour have applications in design exploration, analysis, testing and verification. We describe a new algorithm for automatically extracting useful models, as automata, from execution traces of a HW/SW system driven by software exercising a use-case of interest. Our algorithm leverages modern program synthesis techniques to generate predicates on automaton edges, succinctly describing system behaviour. It employs trace segmentation to tackle complexity for long traces. We learn concise models capturing transaction-level, system-wide behaviour--experimentally demonstrating the approach using traces from a variety of sources, including the x86 QEMU virtual platform and the Real-Time Linux kernel

DeepSynth: Automata Synthesis for Automatic Task Segmentation in Deep Reinforcement Learning

This paper proposes DeepSynth, a method for effective training of deep Reinforcement Learning (RL) agents when the reward is sparse and non-Markovian, but at the same time progress towards the reward requires achieving an unknown sequence of high-level objectives. Our method employs a novel algorithm for synthesis of compact automata to uncover this sequential structure automatically. We synthesise a human-interpretable automaton from trace data collected by exploring the environment. The state space of the environment is then enriched with the synthesised automaton so that the generation of a control policy by deep RL is guided by the discovered structure encoded in the automaton. The proposed approach is able to cope with both high-dimensional, low-level features and unknown sparse non-Markovian rewards. We have evaluated DeepSynth's performance in a set of experiments that includes the Atari game Montezuma's Revenge. Compared to existing approaches, we obtain a reduction of two orders of magnitude in the number of iterations required for policy synthesis, and also a significant improvement in scalability.Comment: Extended version of AAAI 2021 pape

Learning symbolic abstractions from system execution traces

This dissertation shows that symbolic abstractions for a system can be inferred from a set of system execution traces using a combination of Boolean satisfiability and program synthesis. In addition, the degree of completeness of an inferred abstraction can be evaluated by employing equivalence checking using simulation relations, that can further be used to iteratively infer an overapproximating system abstraction with provable completeness guarantees. The first part of this dissertation presents a novel algorithm to infer a symbolic abstraction for a system as a finite state automaton from system execution traces. Given a set of execution traces the algorithm uses Boolean satisfiability to learn a finite state automaton that accepts (at least) all traces in the set. To learn a symbolic system abstraction over large and possibly infinite alphabets, the algorithm uses program synthesis to consolidate trace information into syntactic expressions that serve as transition predicates in the learned model. The system behaviours admitted by the inferred abstraction are limited to only those manifest in the set of execution traces. The abstraction may therefore only be a partial model of the system and may not admit all system behaviours. The second part of this dissertation presents a novel procedure to evaluate the degree of completeness for an inferred system abstraction. The structure of the abstraction is used to extract a set of conditions that collectively encode a completeness hypothesis. The hypothesis is formulated such that the satisfaction of the hypothesis is sufficient to guarantee that a simulation relation can be constructed between the system and the abstraction. Further, the existence of a simulation relation is sufficient to guarantee that the inferred system abstraction is overapproximating. In addition, counterexamples to the hypothesis can be used to construct new traces and iteratively learn new abstractions, until the completeness hypothesis is satisfied and an overapproximating system abstraction is obtained

Active Learning Implementation

Active Learning of Abstract System Models from Traces: An active learning framework for generating concise abstractions of a system implementation, instrumented to track a set of system variables. Generated abstractions admit all system traces defined over the set of system variables

Active learning of abstract system models from traces using model checking

We present a new active model-learning approach to generating abstractions of a system implementation, as finite state automata (FSAs), from execution traces. Given an implementation and a set of observable system variables, the generated automata admit all system behaviours over the given variables and provide useful insight in the form of invariants that hold on the implementation. To achieve this, the proposed approach uses a pluggable model learning component that can generate an FSA from a given set of traces. Conditions that encode a completeness hypothesis are then extracted from the FSA under construction and used to evaluate its degree of completeness by checking their truth value against the system using software model checking. This generates new traces that express any missing behaviours. The new trace data is used to iteratively refine the abstraction, until all system behaviours are admitted by the learned abstraction. To evaluate the approach, we reverse-engineer a set of publicly available Simulink Stateflow models from their C implementations